Detection of hidden structures on all scales in amorphous materials and complex physical systems: basic notions and applications to networks, lattice systems, and glasses

Recent decades have seen the discovery of numerous complex materials. At the root of the complexity underlying many of these materials lies a large number of possible contending atomic- and larger-scale configurations and the intricate correlations between their constituents. For a detailed understanding, there is a need for tools that enable the detection of pertinent structures on all spatial and temporal scales. Towards this end, we suggest a new method by invoking ideas from network analysis and information theory. Our method efficiently identifies basic unit cells and topological defects in systems with low disorder and may analyze general amorphous structures to identify candidate natural structures where a clear definition of order is lacking. This general unbiased detection of physical structure does not require a guess as to which of the system properties should be deemed as important and may constitute a natural point of departure for further analysis. The method applies to both static and dynamic systems.Comment: (23 pages, 9 figures

Multiresolution community detection for megascale networks by information-based replica correlations

We use a Potts model community detection algorithm to accurately and quantitatively evaluate the hierarchical or multiresolution structure of a graph. Our multiresolution algorithm calculates correlations among multiple copies ("replicas") of the same graph over a range of resolutions. Significant multiresolution structures are identified by strongly correlated replicas. The average normalized mutual information, the variation of information, and other measures in principle give a quantitative estimate of the "best" resolutions and indicate the relative strength of the structures in the graph. Because the method is based on information comparisons, it can in principle be used with any community detection model that can examine multiple resolutions. Our approach may be extended to other optimization problems. As a local measure, our Potts model avoids the "resolution limit" that affects other popular models. With this model, our community detection algorithm has an accuracy that ranks among the best of currently available methods. Using it, we can examine graphs over 40 million nodes and more than one billion edges. We further report that the multiresolution variant of our algorithm can solve systems of at least 200000 nodes and 10 million edges on a single processor with exceptionally high accuracy. For typical cases, we find a super-linear scaling, O(L^{1.3}) for community detection and O(L^{1.3} log N) for the multiresolution algorithm where L is the number of edges and N is the number of nodes in the system.Comment: 19 pages, 14 figures, published version with minor change

Local multiresolution order in community detection

Community detection algorithms attempt to find the best clusters of nodes in an arbitrary complex network. Multi-scale ("multiresolution") community detection extends the problem to identify the best network scale(s) for these clusters. The latter task is generally accomplished by analyzing community stability simultaneously for all clusters in the network. In the current work, we extend this general approach to define local multiresolution methods, which enable the extraction of well-defined local communities even if the global community structure is vaguely defined in an average sense. Toward this end, we propose measures analogous to variation of information and normalized mutual information that are used to quantitatively identify the best resolution(s) at the community level based on correlations between clusters in independently-solved systems. We demonstrate our method on two constructed networks as well as a real network and draw inferences about local community strength. Our approach is independent of the applied community detection algorithm save for the inherent requirement that the method be able to identify communities across different network scales, with appropriate changes to account for how different resolutions are evaluated or defined in a particular community detection method. It should, in principle, easily adapt to alternative community comparison measures.Comment: 19 pages, 11 figure

Inference of hidden structures in complex physical systems by multi-scale clustering

We survey the application of a relatively new branch of statistical physics--"community detection"-- to data mining. In particular, we focus on the diagnosis of materials and automated image segmentation. Community detection describes the quest of partitioning a complex system involving many elements into optimally decoupled subsets or communities of such elements. We review a multiresolution variant which is used to ascertain structures at different spatial and temporal scales. Significant patterns are obtained by examining the correlations between different independent solvers. Similar to other combinatorial optimization problems in the NP complexity class, community detection exhibits several phases. Typically, illuminating orders are revealed by choosing parameters that lead to extremal information theory correlations.Comment: 25 pages, 16 Figures; a review of earlier work

Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering

We elaborate on a general method that we recently introduced for characterizing the "natural" structures in complex physical systems via a multiscale network based approach for the data mining of such structures. The approach is based on "community detection" wherein interacting particles are partitioned into "an ideal gas" of optimally decoupled groups of particles. Specifically, we construct a set of network representations ("replicas") of the physical system based on interatomic potentials and apply a multiscale clustering ("multiresolution community detection") analysis using information-based correlations among the replicas. Replicas may be (i) different representations of an identical static system or (ii) embody dynamics by when considering replicas to be time separated snapshots of the system (with a tunable time separation) or (iii) encode general correlations when different replicas correspond to different representations of the entire history of the system as it evolves in space-time. We apply our method to computer simulations of a binary Kob-Andersen Lennard-Jones system, a ternary model system, and to atomic coordinates in a ZrPt system as gleaned by reverse Monte Carlo analysis of experimentally determined structure factors. We identify the dominant structures (disjoint or overlapping) and general length scales by analyzing extrema of the information theory measures. We speculate on possible links between (i) physical transitions or crossovers and (ii) changes in structures found by this method as well as phase transitions associated with the computational complexity of the community detection problem. We briefly also consider continuum approaches and discuss the shear penetration depth in elastic media; this length scale increases as the system becomes increasingly rigid.Comment: (29 pages, 44 figures

The Influence of Network Topology on Sound Propagation in Granular Materials

Granular materials, whose features range from the particle scale to the force-chain scale to the bulk scale, are usually modeled as either particulate or continuum materials. In contrast with either of these approaches, network representations are natural for the simultaneous examination of microscopic, mesoscopic, and macroscopic features. In this paper, we treat granular materials as spatially-embedded networks in which the nodes (particles) are connected by weighted edges obtained from contact forces. We test a variety of network measures for their utility in helping to describe sound propagation in granular networks and find that network diagnostics can be used to probe particle-, curve-, domain-, and system-scale structures in granular media. In particular, diagnostics of meso-scale network structure are reproducible across experiments, are correlated with sound propagation in this medium, and can be used to identify potentially interesting size scales. We also demonstrate that the sensitivity of network diagnostics depends on the phase of sound propagation. In the injection phase, the signal propagates systemically, as indicated by correlations with the network diagnostic of global efficiency. In the scattering phase, however, the signal is better predicted by meso-scale community structure, suggesting that the acoustic signal scatters over local geographic neighborhoods. Collectively, our results demonstrate how the force network of a granular system is imprinted on transmitted waves.Comment: 19 pages, 9 figures, and 3 table

Detecting modules in dense weighted networks with the Potts method

We address the problem of multiresolution module detection in dense weighted networks, where the modular structure is encoded in the weights rather than topology. We discuss a weighted version of the q-state Potts method, which was originally introduced by Reichardt and Bornholdt. This weighted method can be directly applied to dense networks. We discuss the dependence of the resolution of the method on its tuning parameter and network properties, using sparse and dense weighted networks with built-in modules as example cases. Finally, we apply the method to data on stock price correlations, and show that the resulting modules correspond well to known structural properties of this correlation network.Comment: 14 pages, 6 figures. v2: 1 figure added, 1 reference added, minor changes. v3: 3 references added, minor change

Detection of hidden structures for arbitrary scales in complex physical systems

Recent decades have experienced the discovery of numerous complex materials. At the root of the complexity underlying many of these materials lies a large number of contending atomic- and largerscale configurations. In order to obtain a more detailed understanding of such systems, we need tools that enable the detection of pertinent structures on all spatial and temporal scales. Towards this end, we suggest a new method that applies to both static and dynamic systems which invokes ideas from network analysis and information theory. Our approach efficiently identifies basic unit cells, topological defects, and candidate natural structures. The method is particularly useful where a clear definition of order is lacking, and the identified features may constitute a natural point of departure for further analysis

Deciphering Network Community Structure by Surprise

The analysis of complex networks permeates all sciences, from biology to sociology. A fundamental, unsolved problem is how to characterize the community structure of a network. Here, using both standard and novel benchmarks, we show that maximization of a simple global parameter, which we call Surprise (S), leads to a very efficient characterization of the community structure of complex synthetic networks. Particularly, S qualitatively outperforms the most commonly used criterion to define communities, Newman and Girvan's modularity (Q). Applying S maximization to real networks often provides natural, well-supported partitions, but also sometimes counterintuitive solutions that expose the limitations of our previous knowledge. These results indicate that it is possible to define an effective global criterion for community structure and open new routes for the understanding of complex networks.Comment: 7 pages, 5 figure